PR Summary

This PR introduces several key enhancements and refactors across the project:

1. LLM Agent and Notebook Updates:

   • The notebooks now demonstrate an improved LLM-powered tool selection router with enhanced markdown documentation explaining the benefits of intelligent tool routing.
   • The agent functions have been updated to return a structured dictionary that includes the final prediction, raw output, and a list of tools used. Minor formatting changes and refined docstrings make the demo clearer and more instructive.

2. Utility Module Refactoring:

   • The langchain utility module has been streamlined by removing redundant helper functions (such as extra tool extraction and formatting routines) and consolidating functionality to capture only essential tool output data.
   • Import paths have been updated to reflect the new structure, improving code clarity and reducing unused imports.

3. Enhanced Dataset Prediction Assignment:

   • The dataset module now supports diverse prediction outputs including simple values as well as dictionary responses. New helper functions (_handle_deprecated_parameters, _handle_dictionary_predictions, etc.) modularize the process of adding prediction and probability columns.
   • The prediction assignment workflow now properly distinguishes between computed predictions and precomputed prediction values, ensuring correct column naming and consistent data assignment.
   • Comprehensive new unit tests in tests/test_dataset.py cover scenarios for classification, regression, complex dictionary outputs, multiple models, and error-handling (e.g. invalid predict_fn), ensuring robustness of the new functionality.

Overall, these changes enhance the robustness, clarity, and flexibility of both the LLM agent routing demos and the dataset prediction functionality without altering core business logic.

Test Suggestions

• Run all new and existing unit tests in tests/test_dataset.py to ensure all prediction scenarios (classification, regression, complex outputs) are correctly handled.
• Perform integration testing of the LLM agent flow to verify that the tool routing correctly identifies and logs the correct tool usage.
• Simulate edge cases including empty tool outputs and invalid prediction functions to check for proper error handling and warnings.
• Validate that the structured return values (including the 'tools_used' list) are correctly propagated to downstream processing.